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ABSTRACT 

Often data in multi-criteria decision making (MCDM) problems are imprecise and changeable. 
Therefore, an important step in many applications of MCDM is to perform a sensitivity analysis 
on the input data. This paper presents a methodology for performing a sensitivity analysis on the 
weights of the decision criteria and the performance values of the alternatives expressed in terms 
of the decision criteria. The proposed methodology is demonstrated on three widely used 
decision methods. These are the weighted sum model (WSM), the weighted product model 
(WPM), and the analytic hierarchy process (AHP). This paper formalizes a number of important 
issues on sensitivity analysis and derives some critical theoretical results. Also, a number of 
illustrative examples and computational experiments further illustrate the application of the 
proposed methodology. 
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INTRODUCTION 

There is considerable research on sensitivity analysis for some operations research and 
management science models such as linear programming and investment analysis. For example, 
in a sensitivity analysis approach for linear programming, Wendel (1992) utilized a tolerance 
approach to handle variations in the parameters of more than one term (in the LP sense) at a 
time. Furthermore, that type of sensitivity analysis is considered as a post-optimality step. That 
is, the analysis is done after the optimal decision is determined. However, research on sensitivity 
analysis in deterministic multi-criteria decision making (MCDM) models is limited. Myers and 
Alpert (1968) first introduced the notion of the determinant attributes in choice theory and 
consumer behavior. In that development Myers and Alpert focused on consumer and behavioral 
aspects of buyers. Later, Alpert (1971) compared several determinant analysis techniques and 
found significant support for using the direct dual questioning determinant attribute (DQDA) 
approach. That made DQDA popular for marketing applications (see, for instance, Anderson, 
Cox, and Fulcher, 1976; Martin and Winteregg, 1989; and Sinclair and Stalling, 1990). Barron 
and Schmidt (1988) recommended two procedures to accomplish a sensitivity analysis in multi- 
attribute value models. These are an entropy based procedure and a least squares procedure. For 
the entropy based procedure they assumed nearly equal weights. However, the least squares 
procedure required a set of arbitrary weights for the attributes. These procedures calculate for a 
given pair of alternatives, one of which is the best alternative, the closest set of weights that 
equates their ranking. The procedures can also calculate the nearly-equal weights that promote 
the second best alternative by a specific amount to exceed the optimal alternative by a 
predetermined amount. One of their findings is that in additive models, the weights do matter, 
that is, for a small change in the weights the optimal alternative may change. Watson and Buede 
(1987) illustrated sensitivity analysis in a decision modeling strategy. Von Winterfeldt and 
Edwards (1986) covered sensitivity analysis in the traditional way for those problems which can 
be approached by using multi-attribute utility theory (MAUT) or a Bayesian model. They 
defined the Flat Maxima Principle for MAUT problems, which states that the existence of 
dominance makes sensitivity analysis almost unnecessary. Furthermore, they advice against an 
over generalization of flat maxima, which is applicable exclusively to the expected value and 
value functions defined on prior probabilities and weights only. One interesting point of the Von 
Winterfeldt and Edwards approach is the idea of switchover or break-even points. A switchover 
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point is the point at which the parameters produce the break-even utility. Switchovers are 
important because they guide further modeling and elicitation. If the circumstances of the 
problem imply that both the analysis and the parameters one is using are remote from switchover 
points, then the decision maker can be confident of the validity of the current results. 

Evans (1984) investigated linear programming-like sensitivity analysis in decision theory. 
His approach is based on the geometric characteristics of optimal decision regions in the 
probability space. Evans made an analysis on the sensitivity of the optimal decision to changes 
on probabilities of the states of nature. Also, in Triantaphyllou (1992) a sensitivity analysis 
approach is described for a class of inventory models. A methodology for sensitivity analysis in 
multi-objective decision making is described in Rfos Insua (1990). That treatment introduced a 
general framework for sensitivity analysis which expanded results of the traditional Bayesian 
approach to decision making. Emphasis is given to cases which use partial and/or doubtful data. 
Also that work contains an analysis of why the flat maxima principle is not valid. The Rfos Insua 
(1990) book also includes the description of SENSATO; a prototype library of a sensitivity 
analysis package for decision aids. However, the present paper assumes that the data are not 
stochastic and it focuses on the issue of sensitivity analysis on the weights of the decision criteria 
and the performance measures of the alternatives in deterministic environments. Samson (1988) 
presented a whole new approach to sensitivity analysis. He proposed that Sensitivity analysis 
should be part of the decision analysis process thinking in real time. That is, it should be 
integrated into every step of the decision analysis. Samson noted that sensitivity analysis can be a 
most useful tool when it is embedded into a continuous cycle process during which at each stage 
of the decision process the analysis can go back to previous stages to check, add, or modify parts 
of the problem. French (1986), (1989) emphasized the role of sensitivity analysis on decision 
making. He performed an analysis of the use of interactive decision aids to overcome some of 
the difficulties in modeling judgments. The models examined were mostly stochastic as opposed 
to deterministic. Furthermore, he stressed the importance of having better and more general 
sensitivity analysis tools. Also, French and Rfos Insua (1989) used a distance minimization 
approach to determine competitors to a current optimal solution. Some other related sensitivity 
analysis studies are reported in Alexander (1989); Hartog, Hinloopen and Nijkamp (1989); and 
Weber, Eisenfuhr and Von Winterfeldt (1988). 
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A recent development in sensitivity analysis when the analytic hierarchy process (AHP) 
(see Saaty, 1980, 1994) is used is due to Masuda (1990). In that work Masuda studied the effect 
of changes on entire vectors of the decision matrix may have on the ranking of the alternatives. 
That author considered multiple levels of hierarchies. However, he did not offer a procedure for 
performing a sensitivity analysis on changes on an individual piece of data of a given problem 
(i.e., on a single criterion weight or the performance value of an alternatives in terms of a given 
criterion). The proposed sensitivity analysis is Complementary to the one developed by Masuda 
and the two approaches can be used together (since the proposed approach can focus on 
individual judgments while the Masuda approach considers a single vector at a time). Also, 
Armacost and Hosseini (1994) presented a procedure for determining the most critical criterion 
for a single level hierarchy AHP problem. However, the latter work does not explicitly determine 
what is the smallest change on the current weight of a criterion, such that the existing ranking of 
the alternatives will change. As a related comment it should also be stated here that Expert 
Choice (1990), a software package on the AHP, performs a type of elementary sensitivity 
analysis. The user has the option to graphically alter the weights of the decision criteria and see 
on the screen how the rankings of the alternatives will change. However, the issue of criteria 
sensitivity analysis is not studied systematically. Moreover, Expert Choice does not offer any 
means for studying the effects of changes on the measures of performance of the alternatives 
(which is part of the proposed methodology in this paper). In decision making the weights 
assigned to the decision criteria attempt to represent the genuine importance of the criteria. When 
criteria cannot be expressed in quantitative terms (such as cost, weight, volume, etc.), then it is 
difficult to represent accurately the importance of these criteria. In a situation like this, the 
decision making process can be improved considerably by identifying the critical criteria (the 
formal definition is given later) and then re-evaluate more accurately the weights of these 
criteria. The intuitive belief is that the criterion with the highest weight is the most critical one 
(Winston, 1991, p. 754). This may not always be true and in some instances the criterion with the 
lowest weight may be the most critical one. The decision maker can make better decisions if 
he/she can determine how critical each criterion is. In other words, how sensitive the actual 
ranking of the alternatives is to changes on the current weights of the decision criteria. In this 
paper we examine two closely related sensitivity analysis problems. In the first problem we 
determine how critical each criterion is, by performing a sensitivity analysis on the weights of 
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the criteria. This sensitivity analysis approach determines what is the smallest change in the 
current weights of the criteria, which can alter the existing ranking of the alternatives. In the 
second problem, we use a similar concept to determine how critical the various performance 
measures of the alternatives (in terms of a single decision criterion at a time) are in the ranking of 
the alternatives. These two types of sensitivity analysis problems will be explored in latter 
sections. The next section briefly describes the MCDM methods considered in this paper. The 
third section presents the formal definitions of the two sensitivity problems analyzed in this 
study. The related concepts and methods are further illustrated in terms of some demonstration 
examples. Computational experiments were also performed in order to increase the insight of 
these sensitivity issues. Finally, the last section presents the main conclusions of the proposed 
methodology. 

SOME MULTI-CRITERIA DECISION MAKING METHODS 

There are three main steps in utilizing a decision making technique involving numerical 
analysis of a set of discrete alternatives: 

1. Determining the relevant criteria and alternatives. 
2. Attaching numerical measures to the relative importance (i.e., weights) of the criteria and to 
the impacts (i.e, the measures of performance) of the alternatives in terms of these criteria. 

3. Processing the numerical values to determine a ranking of each alternative. 
In this paper we are interested in a sensitivity analysis on the data described in step 2, above. 

Consider a decision making problem with M alternatives and N criteria. In this paper 
alternatives will be denoted as A; (for i = 1,2,3,. ..,M) and criteria as Cj (for j = 1,2,3,. ..,N). We 
assume that for each criterion Cj the decision maker has determined its importance, or weight, 
Wj. It is also assumed that the following relationship is always true: 



N 

ZWj=i 

(i) 

j=i 

Furthermore, it is also assumed that the decision maker has determined a ; j 
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(for i = 1,2,3,. ..,M and j = 1,2,3, ...,N); the importance (or measure of performance) of alternative 
A; in terms of criterion Cj. Then, the core of the typical MCDM problem examined in this paper 
can be represented by the following decision matrix as seen in Table 1 . 
Table 1: Decision matrix. 



Criterion 




c, 


c 2 


C 3 ... 


Cn 














Alt. 


w. 




W 3 ... 


Wn 


Ai 


an 


ai2 


ai3 ••• 


am 








A 2 


a2i 


a22 a23 


... a2N 


A 3 


a 3 i 


a 3 2 a 3 3 . 


- a3N 








A M 


aMi 


aM2 aM3 •• 


. aMN 



Some decision methods (for instance, the AHP) require that the ay values represent relative 
importance. Given the above data and a decision making method, the objective of the decision 
maker is to find the best (i.e., the most preferred) alternative or to rank the entire set of 
alternatives. Let P; (for i = 1,2,3, ...,M) represent the final preference of alternative A; when all 
decision criteria are considered. Different decision methods apply different procedures in 
calculating the values Pi. Without loss of generality, it can be assumed (by a simple 
rearrangement of the indexes) that the M alternatives are arranged in such a way that the 
following relation (ranking) is satisfied (that is, the first alternative is always the best alternative 
and so on): 

Pi> P 2 > P 3 ... > Pm . 

(2) 
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The next subsections briefly describe the three MCDM methods and one variant to be considered 
in this paper. For an evaluation of these methods the interested reader may want to consult with 
the analyses reported in Triantaphyllou (1989, 1994). 
The Weighted Sum Model 

Probably the simplest and still the widest used MCDM method is the weighted sum 
model (WSM). The preference Pi of alternative A; (i = 1, 2, 3,..., M) is calculated according to 
the following formula (Fishburn, 1967): 
N 

Pi=Z aijWj=l , i =1,2,3 M 

(3) 
j=l 

Therefore, in the maximization case, the best alternative is the one which corresponds to the 
largest preference value. The supposition which governs this model is the additive utility 
assumption. However, the WSM should be used only when the decision criteria can be expressed 
in identical units of measure (e.g., only rupees, or only kgs, or only seconds, etc.). 
The Weighted Product Model 

The weighted product model (WPM) is very similar to the WSM. The main difference is that 
instead of addition in the model there is multiplication. Each alternative is compared with the 
others by multiplying a number of ratios, one for each criterion. Each ratio is raised to the power 
equivalent to the relative weight of the corresponding criterion. In general, in order to compare 
alternatives A p and A q (where M > p, q > 1) the following product (Bridgman, 1922; Miller and 
Starr, 1969; Chen and Hwang, 
1992) has to be calculated: 
N 

R(Ap/A q )= n(a pj / atlj ) wj 



J=l 



(4) 



If the ratio R(A P /A q ) is greater than or equal to one, then the conclusion is that alternative 
A p is more desirable than alternative A q (for the maximization case). The best alternative is the 
one which is better than or at least equal to all other alternatives. The WPM is sometimes called 
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dimensionless analysis because its structure eliminates any units of measure. Thus, the WPM 
can be used in single and multi-dimensional decision making problems. 
The Analytic Hierarchy Process 

Part of the analytic hierarchy process (AHP) (Saaty, 1980, 1994) deals with the structure of an 
MxN matrix. This matrix, say matrix A, is constructed using the relative importance of the 
alternatives in terms of each criterion. The vector (a^i, a;,2, a^, ...,ai N ), for each i=l,2,...,M, in 
this matrix is the principal eigenvector of an NxN reciprocal matrix which is determined by pair 
wise comparisons of the impact of the M alternatives on the i-th criterion. Some evidence is 
presented in Saaty (1980) which supports the technique for eliciting numerical evaluations of 
qualitative phenomena from experts and decision makers. According to AHP the final 
preference, P;, of alternative A; is also given by formula (3). However, now the ay value 
expresses the relative performance value of alternative A; when it is examined with the rest of 
the other alternatives in terms of criterion Cj. In the maximization case, the best alternative is the 
one which corresponds to the highest Pi value. The similarity between the WSM and the AHP is 
clear. The AHP uses relative values instead of absolute measures of performance (which may or 
may not be readily available). In the original version of the AHP the performance values ay are 
normalized so they sum up to one. That is, the following relation is always true in the AHP case: 
M 

£aij= 1 for j =1,2,3. ...N 
(5) 
i=l 

Thus, it can be used in single or multi-dimensional decision making problems. 

Belton and Gear (1983) proposed a revised version of the AHP model. They 
demonstrated that an unacceptable rank reversal may occur when the AHP is used. Instead of 
having the relative values of the alternatives Ai, A2, A3, A M sum up to one (e.g. equation (5) 
to hold), they propose to divide each relative value by the maximum quantity of the relative 
values in each column of the MxN matrix A. Later, Saaty (1994) accepted the previous notion as 
a variant of the original AHP and now he calls it the ideal mode AHP. 
DESCRIPTION OF THE TWO MAJOR SENSITIVITY ANALYSIS PROBLEMS 
The structure of the typical decision problem considered in this paper consists of a number, say 
M, of alternatives and a number, say N, of decision criteria. Then, the pertinent data form a 
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decision matrix as described earlier at the beginning of the second section of this paper. Given 
such a decision matrix, the decision problem considered in this paper is how to determine which 
the best alternative is or rank the entire set of alternatives. In a simple MCDM situation, all 
criteria are expressed in terms of the same unit (e.g., Rupees). However, in many real life 
MCDM problems different criteria may be expressed in different dimensions (units). Examples 
of such dimensions include Rupee figures, weight, time, political impact, environmental impact, 
etc. It is this issue of multiple dimensions which makes the typical MCDM problem to be a 
complex one. Given the above data, the objective of the decision maker is to rank the 
alternatives. The alternatives are ranked according to their final preferences P; (i=l, 2,3,. ..,M). 
Recall that the P; values are calculated according to formulas (3), (4) and (5). 

The first major problem which is examined in this paper is how to determine the most 
critical criterion in the previous decision making problem. Intuitively, one may think that the 
most critical criterion is the criterion which corresponds to the highest weight Wj. However, 
this notion of criticality may be misleading. In this paper, the most critical criterion is defined in 
two alternative ways. In the first way the interest is on whether the indication of the best (top) 
alternative changes or not. On the second definition the interest is on changes on the ranking of 
any alternative. These definitions are given formally in the next section. In the previous notion of 
criticality, the term smallest change can be defined in two different ways. The first way is to 
define smallest change in absolute terms. The second way is to define smallest change in 
relative terms. For instance, suppose that the two criteria Ci and C2 have weights Wi = 0.30 and 
W2 = 0.50, respectively. Furthermore, suppose that when the first weight becomes W 1 ' = 0.35, 
then the existing ranking of the alternatives changes. Similarly, suppose that when the second 
weight becomes W 2 ' = 0.57, then the existing ranking of the alternatives changes. In absolute 
terms for both criteria, the first criterion is the most critical criterion. This is true since the 
change of the weights for d is: *Wi -Wi'*= 0.05, while for C 2 it is: *W 2 -W 2 '* = 0.07. That is, 
for the first criterion the critical change is smaller than for the second criterion. However, when 
one considers relative terms, then the previous picture changes. In relative terms, the change of 
the weights for d is: *Wi -Wi' *xl00/Wi = 16.67, while for C 2 it is: *W 2 -W 2 *xl00/W 2 = 
14.00. That is, for the second criterion the relative change is smaller than for the first criterion. 
Therefore, when the relative changes are considered, then the most critical criterion is C2. It can 
be observed that regarding changes on the ranking of the alternatives one may view them from 
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two different perspectives as follows. First, one might be interested to see when a change in the 
current data causes any two alternatives to reverse their existing ranking. However, it is also 
possible one to be interested only when the best (top) alternative changes. 

Therefore, a total of four alternative definitions can be considered. These are coded as 
Absolute Any (AA), Absolute Top (AT), Percent Any (PA), and Percent Top (PT). This 
approach, however, might be misleading. After all, a change, say by 0.03, does not mean much 
unless someone is also given the original value. A change of 0.03 is very different if the original 
value was 0.08 or 0.80. That is, it is more meaningful to use relative changes. Therefore, in this 
paper the emphasis will be on relative (percent) changes and thus all developments are based on 
relative changes. However, the proposed methodology and illustrative numerical examples 
presented later, do present how one can also derive changes in absolute terms. It can be noticed 
that in order for one to derive changes in relative terms, changes in absolute terms need to be 
calculated first. The above notion of critical change is used to determine both the most critical 
criterion (problem-1) and the most critical aij performance measure (problem-2). As an 
extension of determining the most critical criterion (or aij performance measure), the notion of 
critical change is used to determine how critical each criterion weight W; and performance 
measure a;j is. These two problems are examined in more detail in the subsequent sections. 
PROBLEM 1: DETERMINING THE MOST CRITICAL CRITERION 
Definitions and Terminology 

First we consider the case of changes in the current weights of the decision criteria. 
DEFINITION 1: Let 5 kii j (1 < i < j < M and 1 < k < N) denote the minimum change in the 
current weight W\ of criterion Ck such that the ranking of alternatives A; and Aj will be reversed. 
Also, define as: 
8 Vu = 5 k ,ij x 100/W k , for any 1 < i < j < M and 1 < k < N. 
(6) 

That is, 8' k.i,j expresses changes in relative terms. As it will be shown later (in Theorem 1) it is 
possible for a given pair of alternatives and a decision criterion, the critical change to be 
infeasible. 

The most critical criterion is defined in two possible ways (recall that from relations (2) 
alternative Ai is always assumed to be the best alternative). The first of these two definitions 
(i.e., definition 2) applies when one is interested only in changes in the best alternative, while 
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the second definition (i.e., definition 3) applies when one is interested in changes in the ranking 
of any alternative. Recall that Isl stands for the absolute value function (e.g., 1-51 = +5). 
DEFINITION 2: The Percent-Top (or PT) critical criterion is the criterion which 
corresponds to the smallest 15' k,i,j I ( 1 < j < M and 1 < k < N) value. 

DEFINITION 3: The Percent-Any (or PA) critical criterion is the criterion which 
corresponds to the smallest |8' k,i,j | ( 1 < j < M and 1 < k < N) value. 

It can be recalled that in this paper we adopt the definitions which correspond to relative 
changes. The following two definitions express how critical a given decision criterion is. 
DEFINITION 4: The criticality degree of criterion Ck, denoted as D' k , is the smallest percent 
amount by which the current value of Wk must change, such that the existing ranking of the 
alternatives will change. That is, the following relation is true: 
D' k = min |8' MJ I foraUN>k>l. 
1< i< j < M 

DEFIN ITION 5 The sensitivity coefficient of criterion Ck, denoted as sens(Ck), is the 
reciprocal of its criticality degree. That is, the following relation is true: 
sens(C k ) = 1/ D'k, for any N > k >1. 

If the criticality degree is infeasible (i.e., impossible to change any alternative rank with any 
weight change), then the sensitivity coefficient is set equal to zero. 

The previous two definitions 4 and 5 are based on changes on the ranking of any 
alternative. However, one may be interested only in changes on the ranking of the best (top) 
alternative. For instance, in a problem involving the purchase of a house, the focus is on the best 
house and the ranking of all alternative houses may be of secondary interest. In cases like the 
above, one may want to use modifications of the criticality degree and sensitivity coefficient 
concepts in which changes are only defined on the ranking of the best alternative. Also observe 
that since D' k is always less than 1.00, it follows that the value of sens(Ck) is always greater 
than or equal to 1 .00. 

Some Theoretical Results in Determining the Most Critical Criteria 
Case (i): Using the WSM or the AHP Method 

Now it is assumed that a decision maker uses the WSM or the AHP method and he/she wishes to 
alter the existing ranking of alternatives Ai and Aiby modifying only the current weight Wl of 
criterion Ci. Currently, the following relation is true (as it was assumed in (2)): Pi > P2. In the 
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Appendix it is shown that the minimum quantity 5 1,1,2, needed to reverse the current ranking of 
alternatives Ai and A2, should satisfy the following relation: 
(P2-P1) 

811,2 < ifa2i>an or: 



(a 2 i-an) 



(7a) 



(P2-P1) 



'1,1,2 



if a2i < an or: 



(a 2 i-an) 

Furthermore, the following condition should also be satisfied for the new weight 
Wi* = Wi - 8 1,1,2 to be feasible: 
< Wi , which implies 

< Wi - 8 1,1,2 , which implies (7b) 
8 1,1,2 < Wi . 

In these developments it is not required to have W i * < 1 because these weights are 
re-normalized to add up to one. From relations (7a) and (7b), above, it can be seen that 
sometimes the value 8 u,2 may not have a feasible value. In other words, it may be impossible 
to reverse the existing ranking of the alternative Ai and A2 by making changes on the current 
weight of criterion CI. This situation occurs when the value of the ratio: 

( P2 - Pi ) / (a2i -an) is greater than Wi. The previous considerations can be generalized easily 
and thus lead to the proof of the following theorem which covers the general case (recall that 
currently the following relation is assumed to be true from (2): P; > Pj, for all 1 < i < j < M). 
THEOREM 1: When the WSM, AHP, or ideal mode AHP methods are used, the quantity 8 'k,ij 
(1 < i < j < M and 1 <k <N), by which the current weight Wk of criterion Ck needs to be 
modified (after normalization) so that the ranking of the alternatives A; and Aj will be reversed, 
is given as follows: 



(Pj-Pi) 100 



if a jk >a ik or: 



(a jk -a ik ) W k 



(8a) 
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8'WJ > 



(Pj-Pi) ioo 



(ajk-a ik ) W k 



if a jk < a ik or: 



Furthermore, the following condition should also be satisfied for the value of S'kjj to be feasible: 
(Pj-Pi) 

< W k 

(8b) (a jk -a ik ) 

From the previous considerations it can be seen that if alternative Ai dominates 
alternative Aj (i.e., a;k > ajk, for all k =1, 2, N) then, it is impossible to make alternative Aj 
more preferred than alternative Ai by changing the weights of the criteria. Also, a criterion Ck is 
a robust criterion if all 8'k,ij (for 1 < i < j < M and 1 < k <N) quantities associated with it are 
infeasible. In other words, if equation (8b) is violated for all i,j = 1,2,3,. ..,M, for some criterion 
Ck then, any change on the weight of that criterion does not affect the existing ranking of any of 
the alternatives and thus this criterion is a robust one and consequently it can be dropped from 
further consideration. Therefore, if one is interested in determining the most critical criterion, 
then all possible 8'k,ij (for 1 <i < j < M and 1 < k <N) values need to be calculated. Observe that 
there are Nx(M(M-l))/2 such possible S'k,ij quantities. This issue is further illustrated in the 
following numerical example. 
A Numerical Example for the WSM Case 

Consider a decision making problem with the four alternatives Ai, A2, A3, and A4 and the four 
decision criteria Ci, C2, C3, and C4. Suppose that the following Table 2 is its corresponding 
decision matrix when the WSM (or the AHP with one hierarchical level) is used. Note that the 
data were normalized to add up to one, although this is not required by WSM (however, it is 
required by the AHP). 

Table 2: Decision Matrix for the numerical example on the WSM. 



Criterion 
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Ci 












Alt 0.3277 


0.3058 


0.2876 


0.0790 


Ai 0.3088 
0.1922 


0.2897 


0.3867 


A 2 0.2163 
0.6288 


0.3458 


0.1755 


A 3 0.4509 
0.0575 


0.2473 


0.1194 


A 4 0.0240 
0.1215 


0.1172 


0.3184 



Suppose that we want to apply the WSM (the case of the AHP is identical since the data are 
already normalized). Then, by using formula (3)) the final preferences and ranking of the four 
alternatives are as shown in Table 3. 

Table 3: Current final preferences. 



Alternative 


Preference (Pj) 


Ranking 


Ai 


0.3162 


1* 




A 2 


0.2768 


2 




A 3 


0.2621 


3 




A 4 


0.1449 


4 





Note: * indicates the most preferred (best) alternative. 

Therefore, the relation Pi > P2 > P3 >P4 holds and as result the most preferred alternative 

is Ai. 

Observe that according to the weights of the four criteria, criterion d appears to be the most 
important one. The minimum change 5 1,1,3 needed to alter the current weight Wi so that the 
current ranking of the two alternatives Ai and A3 will be reversed, can be found by using 
relation (8a) of Theorem 1 as follows: 
( 0.2621-0.3162) 
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81,13 < or 

(0.4509 - 0.3088) 
8 1,1,3 < - 0.3807. 

The quantity - 0.3807 satisfies (8b), as it is less than Wi (= 0.3277). Thus, the modified weight 
Wi* of the first criterion (before normalization) for this case is: Wi* = [ 0.3277 - (-0.3807)] = 
0.7084. 

Working as above for all possible combinations of criteria and pairs of alternatives, Table 4 is 
derived. Table 5 depicts the changes in relative terms (that is, the 8' k,i,j values as computed by 
using relation (8a) of Theorem 1). Observe that negative changes in Table 4 indicate increases, 
while positive changes indicate decreases. Also note that the changes (either percentages or in 
absolute terms) are before normalization. The boldfaced numbers in either table indicate 
minimum critical changes (as explained in the next paragraphs). 

The Percent-Top (PT) critical criterion can be found by looking for the smallest 
relative value of all rows which are related to alternative Ai (i.e., the best alternative) in Table 5. 
The smallest such percentage (i.e., 64.8818%) corresponds to criterion C3 when the pair of 
alternatives Ai and A2 is considered. For criterion C3 a reduction of its current weight by 
64.8818% will make A 2 the most preferred alternative and Ai will not be the best alternative any 
more. 



Table 4: All possible 8k,ij values (absolute change in criteria weights). 
Criterion 



Pair of 
Alternatives 


Ci 


c 2 


c 3 


c 4 


A1-A2 


N/F 


-0.7023 


0.1866 


-0.0902 


AI - A3 


-0.3807 


N/F 


0.2024 


N/F 










A1-A4 


N/F 


N/F 


N/F 


N/F 


A2 - A3 


-0.0627 


0.1492 


0.0262 


0.0257 


A2 - A4 


N/F 


N/F 


-0.9230 


N/F 


A3 - A4 


.2745 


N/F 


-0.5890 


-1.8313 



Note: N/F stands for Non-Feasible. That is, the corresponding 8 value does not satisfy 
relation (8b). 
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The Percent-Any (PA) critical criterion can be found by looking for the smallest relative 
8'k,ij value in the entire Table 5. Such smallest value is 5' 3,2,3 = 9.1099% and it (again) 
corresponds to criterion C3. Therefore, the PA critical criterion is C3. Finally, observe that it is 
a coincidence that both definitions of the most critical criterion indicate the same criterion (i.e., 
criterion C3) in this numerical example. At this point it should be stated that if a decision maker 
wishes to define the most critical criterion in absolute changes, then the previous two definitions 
of Percent-Top (PT) and Percent-Any (PA) critical criterion correspond to Absolute-Top (AT) 
and Absolute-Any (AA) critical criterion, respectively. From Table 4 it can be easily verified 
that the AT criterion is C4 and also, by coincidence, the AA criterion is 

C4 (the corresponding minimum changes are boldfaced). Later, some computational results 
indicate how frequently various alternative definitions of the most critical criterion may point out 
to the same criterion. When definition 4 is used, then from Table 5 it follows that the criticality 
degrees of the four criteria are: Di' = 119.13341 = 19.1334, D 2 ' = 48.7901, D 3 ' = 9.1099, and D 4 ' 
= 32.5317. Therefore, the sensitivity coefficients of the four decision criteria (according to 
definition 5) are: sens (d) = 0.0523, sens (C 2 ) = 0.0205, sens (C 3 ) = 0.1098, and sens (C 4 ) = 
0.0307. That is, the most sensitive decision criterion is C3, followed by Ci, C4, and C2. 

Table 5: All possible 5'k,i j values (percent change in criteria weights). 



Pair of 


Criterion 


Alternatives 


Ci 


c 2 


c 3 


c 4 












Ai - A 2 


N/F 


-229.7 


64.8818 


-114.1772 


Ai-A 3 


-116.1733 


N/F 


70.3755 


N/F 


Al - A4 


N/F 


N/F 


N/F 


N/F 






A2 - A3 


-19.1334 


48.7901 


9.1099 


32.5317 


A2 - A4 


N/F 


N/F 


-320.9 


N/F 


A3-A4 


83.7656 


N/F 


-204.8 


-2,318.10 



Case (ii): Using the WPM Method 



A Monthly Double-Blind Peer Reviewed Refereed Open A ccess Interna tional e-Journal - Included in the International Serial Directories 
Indexed & Listed at: Ulrich's Periodicals Directory ©, U.S.A., ItMJiPBtf^f j as well as in Cabell's Directories of Publishing Opportunities, U.S.A. 



International Journal of Management, IT and Engineering 
http : / /www. i j mra. us 



August 
2013 



Volume 3, Issue 8 



ISSN: 2249-0558 



Suppose that we are interested in determining the quantity 8' 1,1,2 when the WPM method is used. 
Recall that according to relation (4) alternative Ai is more preferred than alternative A2 when 
the following ratio is greater than or equal to one: 
N 

RCAi/Aa) = n ( aij /a 2j ) wj 
(9) 

j=l 

Furthermore, according to (2), it is currently assumed that Pi >P2. Let PI' and P2' denote the 
new preferences of the two alternatives. Then, when the ranking of these two alternatives is 
reversed, the relation on the preferences becomes: PI' < P2'. In the Appendix it is shown that 
the quantity 5 1,1,2 must satisfy the following condition: 
N 

Log ( n( aly /a 2y ) w y ) 



y=i 



8 ' 



(10) 

log ((an/a 2 i) 

The last relationship gives the minimum quantity needed to modify the current weight 
Wl of criterion Ci such that alternative A2 will become more preferred than alternative Al (in 
the maximization case). Similarly as in the previous subsection, this quantity needs to satisfy 
condition (7b). The previous considerations can be easily generalized and thus lead to the proof 
of the following theorem: 

THEOREM 2: When the WPM method is used, the critical quantity 5 ' k ,i,j (1 < i < j < M and 1 
<k <N) by which the current weight Wk of criterion Ck needs to be modified (after 
normalization) so that the ranking of the alternatives Ai and Aj will be reversed, is given as 
follows: 



8 'k,y > k if k > ,or 8 < k otherwise 
where K is defined as: 
N 

log( n (a iy /a jy ) w y ) 

A Monthly Double-Blind Peer Reviewed Refereed Open A ccess Interna tional e-Journal - Included in the International Serial Directories 
Indexed & Listed at: Ulrich's Periodicals Directory ©, U.S.A., ItMJiPBtf^f j as well as in Cabell's Directories of Publishing Opportunities, U.S.A. 

International Journal of Management, IT and Engineering 

http://www.ijmra.us 



156 




Furthermore, the following constraint should also be satisfied: 

8 \y < 100 
(lib) 

Similarly to Theorem 1, in order to determine the most critical criterion a total of 

2 (N x M(M - l)/2) critical changes (i.e., 8 values) need to be calculated. All previous 

theoretical considerations for the WPM model are further illustrated in the following numerical 

example. 

A Numerical Example for the WPM Case 

Consider a decision making problem with the four alternatives Ai, A2, A3, and A4 and the four 
decision criteria CI, C2, C3, and C4. Please note that this numerical example is different than the 
first one (and the two examples which follow), in order to provide a wider exposure of numerical 
scenarios. Also, the decision matrices are square (i.e., M = N) of mere coincidence. The 
proposed procedures can be applied to any size of decision matrix without any modification at 
all. Next, suppose that Table 6 depicts the decision matrix for this numerical example. 
Table 6: Decision matrix for the numerical example on the WPM. 
Criterion 
CI C2 C3 C4 

Alt. 0.4504 0.1231 0.0848 
0.3417 

~A1 0.9381 0.3501 0.8811 
0.5646 

~A2 0.7691 0.4812 0.1679 
0.9336 

~A3 0.9445 0.1138 0.2219 
0.0135 

~A4 0.1768 0.0221 0.9462 
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0.1024 



Then, by applying the WPM approach (that is, by using formula (4)) the ranking of the 
four alternatives is as shown in Table 7. The product expressed by (4) for alternative Al is 
greater than one for all possible combinations which include Ai, thus, the most preferred 
alternative is Al. Also, according to the weights of the four criteria, criterion Ci appears to be 
the most important one, because this is the criterion with the highest weight. 

Table 7: Current ranking. 



Pair of 
Alternatives 
(Ai-Aj ) 


(Ai/Aj) 
Ratio 


Ranking 


A1-A2 


1.0192 


Al 1* 




A1-A3 


4.6082 


A2 2 




Al- A4 


5.3062 


A3 3 




A2- A3 


4.5216 


A4 4 


A2 - A4 


5.2065 




A3 - A4 


1.1515 





Note: * indicates the most preferred alternative (in the maximization case). 
Consider, the minimum quantity needed to alter the current weight W4, so that the current 
ranking of the two alternatives Al and A2 will be reversed. This quantity (expressed as %) can 
be found by using relation (1 la) of Theorem 2 as follows: 



Log((0.938 1/0.7692) 04504 ((0.3501/0.4812) 01231 ((0.5646/0.9336) 03417 xl00/0.3417 



K: 



Lo^fO.5646/0.9336) 

= -11.04, 

therefore, the value of 8 '4,1,2 should be smaller than K = - 1 1.04. Note that this is a feasible value 
since it can easily be verified that it satisfies the constraint given as (lib). In a similar manner, 
all possible K values can be determined. These values are depicted in Table 8 (the boldfaced 
number corresponds to the minimum change). 

It is interesting to observe that the PT, and PA critical criteria happened to point to the 
same criterion, (i.e., criterion C4). However, the criterion with the highest weight is criterion CI. 
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Clearly, this is a counter-intuitive conclusion. Also, when definition 4 is used, then from Table 
8 it follows that the criticality degrees of the four criteria are: D' i = 18.69, D' 2 = 1-48.441 = 
48.44, D' 3 = 13.50, and D' 4 = 1-11.041 = 11.04. Therefore, the sensitivity coefficients of the four 
decision criteria (according to definition 5) are: sens(Cl) = 0.0535, sens(C2) = 0.0206, sens(C3) 
= 0.0741, and sens(C4) = 0.0906. That is, the most sensitive decision criterion is C4, followed 
by C3,Cl,andC2. 



Table 8: All possible K values for WPM example. 



Pair of 
Alternatives 


Criterion 


CI 


C2 


C3 


C4 


Al 




- A2 


21.21 


-48.44 


13.50 


-11.04 


Al 




- A3 


N/F 


N/F 




N/F 


N/F 


Al 




- A4 


N/F 


N/F 


N/F 


N/F 




A2 




- A3 




N/F 


N/F 




] 


S/F 


N/F 




A2 




- A4 




N/F 


N/F 






] 


S/F 


N/F 




A3 




- A4 




18.69 


69.97 








■114.72 


-20.37 



Note: N/F stands for non-feasible, i.e., the corresponding value does not satisfy constraint (13b). 
SOME COMPUTATIONAL EXPERIMENTS 

A computational study was undertaken to study how often the PT and PA critical criteria were 
the criteria with the highest or with the lowest weight. For that reason, random decision 
problems were generated and the PT and PA critical criteria were determined. For the AHP case 
(only) the data for these problems were generated in a manner similar to the procedure used in 
Triantaphyllou, Pardalos and Mann (1990a), Triantaphyllou, Lootsma, Pardalos and Mann 
(1994); and Triantaphyllou and Mann (1990). This manner ensures that the problems are 

A Monthly Double-Blind Peer Reviewed Refereed Open A ccess Interna tional e-Journal - Included in the International Serial Directories 
Indexed & Listed at: Ulrich's Periodicals Directory ©, U.S.A., ItMJiPBtf^f j as well as in Cabell's Directories of Publishing Opportunities, U.S.A. 

International Journal of Management, IT and Engineering 

http://www.ijmra.us 



159 



August 
2013 



Volume 3, Issue 8 



ISSN: 2249-0558 



generated completely randomly. For the WPM and WSM cases the data were generated 
randomly from the uniform distribution in the interval [1,9]. 

According to the test problem generation approach described in Triantaphyllou, Lootsma, 
Pardalos and Mann (1994) the data were generated as follows. First a random weight vector W 
was generated such that the ratio of the largest to the smallest element was less than 9 (in order 
to comply with the values in the Saaty scale). From these weights the entries of the matrix with 
the actual pair wise comparisons were determined by using the relationship aij = wi/wj. It is 
assumed that the decision maker does not know these values. This matrix is called in 
Triantaphyllou, Lootsma, Pardalos and Mann (1994) the Real and Continuous Pairwise (RCP) 
matrix. However, it is assumed that the decision maker is capable of estimating the entries of the 
RCP matrix by forming a matrix in which each entry in the RCP matrix is replaced by a number 
which is as close as possible to the values allowed in the traditional Saaty scale (i.e., the 
numbers from the set {9, 1, 1/2, 1/9}). This is called the Closest and Discrete Pair wise 
(CDP) matrix. Next, the eigenvector of the CDP matrix is estimated and the corresponding 
vector of the decision matrix is formed. 

For instance, if the real (and hence unknown) performance values of three alternatives in 
terms of a single criterion are: (0.77348, 0.23804, 0.23848), then, the (1,3) element of the 
corresponding RCP matrix is equal to 3.24342 (= 0.77348/0.23848). Thus, the corresponding 
CDP element will be equal to 3 (because this value is the closest one from the Saaty scale 
values: {9, 1, 1/2, 1/9}). More on this approach and some interesting properties of the CDP 
matrices can be found in Triantaphyllou, Lootsma, Pardalos and Mann (1994). Two parameters 
were considered in these test problems. The first parameter was the number of decision criteria. 
The second parameter was the number of alternatives. The number of decision criteria was equal 
to 3, 5, 7, 21. Similarly, the number of alternatives was equal to 3, 5, 7, 21. In this way we 
formed 100 different combinations of numbers of criteria and alternatives and 1,000 random test 
problems were generated for each such combination. This simulation program was written in 
Fortran, using the EVISL library of subroutines for generating random numbers. These results are 
depicted in Figures 1 to 12. 

For each test problem we examined whether the PA or the PT critical criterion was the 
criterion with the highest or the criterion with the lowest weight. The results of the 
computational experiments, when the relative (percent) changes are considered, are depicted in 
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Figures 1 to 4. Figures 5 to 8 illustrate the same concepts but when changes are expressed in 
absolute terms. Finally, Figure 9 depicts some specific results when the WPM is used. In the 
present study, we solved each problem using the WSM, WPM, AHP and ideal mode AHP 
method. The four curves in each figure represent the results from each one of the three different 
MCDM methods used plus one curve for the ideal mode AHP. The most profound observation 
is that all MCDM methods generated almost identical results. This is indicated by the fact 
that their curves in Figures 1 to 8 are very close to each other. Figures 1 to 8 indicate that it 
makes a significant difference whether critical changes are expressed as percent (i.e., in relative 
terms) or in absolute terms. When changes are expressed as percentages , then more frequently 
the criterion with the highest weight is the most critical criterion. This is true both when the 
concept of the critical criterion is defined in terms of changes on the ranking of the top 
alternative or in terms of changes on the ranking of any alternative. This is evident when one 
compares Figure 1 with Figure 2 and Figure 3 with Figure 4. The reverse situation occurs when 
one defines change in absolute changes. That is, now more frequently the most critical criterion 
is the criterion with the lowest weight. Figures 5 to 8 depict the corresponding results. As 
anticipated, the sensitivity importance of any weight (including highest or lowest) reduces 
gradually as the number of decision criteria in a problem increases. In a matter of fact, when 
changes are measured in relative terms (i.e., as a percentage), then the lowest weight is hardly 
ever sensitive in problems with more than 10 criteria (see also Figures 2 and 4). On the other 
hand, the number of alternatives has only a minor practical influence. This is indicated in 
Figures 11 and 12 in which the bottom curve corresponds to problems with 3 alternatives and 
the top curve to problems with 21 

alternatives. The question which is raised at this point is what kind of changes a decision maker 
should consider: The ones defined as percentages or the ones defined in absolute terms? 

One may argue here that percentage changes are the most meaningful. After all, a change, say of 
0.03, does not mean much unless one also considers what was the initial value (for instance, was 
the initial value equal to 0.95 or to 0.05?). Figure 9 depicts how frequently the AT and PT 
definitions pointed out to the same criterion. Please recall that this situation also occurred in 
some of the illustrative examples analyzed earlier. Similarly, Figure 10 depicts how frequently 
the AA and PA definitions pointed out to the same criterion. As expected, the frequency of 
matching the top rank is always higher than matching all ranks. Moreover, this distinction fades 
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away as the number of criteria in a problem increases. Finally, Figure 11 depicts how frequently 
all alternative definitions (i.e., AT, PT, AA, and PA) pointed out to the same criterion when the 
WSM model was used (the other models yielded similar results). 
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These computational results indicate that the previous coincidence rates are rather high (around 
70 to 50 or 40 %) when the number of decision criteria in a problem is rather small (less than 7 
or 9). Therefore, if the number of decision criteria is small, one may not have to be concerned on 
which definition to use. Finally, notice that the number of alternatives in a test problem did not 
seem to be important. This is indicated by the closeness of the curves (which correspond to 
problems with different numbers of alternatives) in Figure 12 (which shows a particular set of 
results when the WPM is used, the other methods yielded similar patterns). 
PROBLEM 2: 

DETERMINING THE MOST CRITICAL a y MEASURE OF PERFORMANCE 
Definitions and Terminology 

The second major problem examined in this paper is how to determine the most critical aij 
measure of performance when the WSM, AHP, or the WPM method is used. The following 
definitions are pertinent to this problem. 

DEFINITION 6 Let ijik (1 < i < k < M and 1 < j <N) denote the threshold value of aij, which 
is the minimum change which has to occur on the current value of aij such that the current 
ranking between alternatives Ai and At will change. 
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Since there are M alternatives, each aij performance measure is associated with a total of 
(M-l) such threshold values. In a similar way as earlier regarding the definition of the 5'k,ij 
values, one can also consider threshold values expressed in relative terms. We denote these 
relative term threshold values as ' ij,k 

That is: ' i j,k. = 'i d , k x 100/aij, for any 1 < i, k <M, and 1 < j < N. 
(12) 

For the reasons explained earlier, when we consider threshold values we will mean the ones 
defined in relative terms (i.e., the ' y,k values). Given the previous notion of threshold value, 
we define as the most sensitive alternative the one which is associated with the smallest 
threshold value. Also as before, one may be interested in changes of the ranking of (only) the 
best alternative, or in changes in the ranking of any alternative. As it was mentioned in a 
previous paragraph, there are (M-l) possible threshold values 'i,j,k (i.k, 1 < i, k <M, and 1 < j < 
N) for any aij measure of performance (how to calculate these threshold values is given later in 
theorems 3, 4, and 5). The following three definitions are related to the previous notion of 
threshold values i,j,k. It can be noticed that analogous definitions are possible if one is interested 
in changes only on the top alternative (as was the case earlier with definitions 4 and 5). 
DEFINITION 7: The criticality degree of alternative Ai, denoted as ) 'ij, in terms of criterion 
Cj is the smallest amount (%) by which the current value of aij must change, such that the 
existing ranking of alternative Ai will change. That is, the following relation is true: 
' ij = min(l' ij,kl) , for all M > i > 1, and N > j > 1. 

(13) 

k=i 

DEFINITION 8: Alternative AL is the most critical alternative if it is associated with the 
smallest criticality degree. That is, if and only if the following relation is true: 
'Lk = min (min,('ij)) for some N > k > 1. 

(14) 
M>i>l N>j>l 

DEFINITION 9 The sensitivity coefficient of alternative Ai in terms of criterion Cj, denoted 
as sens(aij), is the reciprocal of its criticality degree. That is, the following condition is true: 
sens(ajj ) =l/'jj ' , for any M > i > 1, and N > j > 1. 
(15) 
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If the criticality degree is infeasible, then the sensitivity coefficient is set equal to zero. 

From definition 7 it follows that the smaller the criticality degree ) \j is, the easier the 
ranking of alternative Ai can change. Alternatively, definition 9 indicates that ranking changes 
are easier, as the sensitivity coefficients sens(aij) are higher. Finally, definition 8, when 
combined with definitions 7 and 9, indicates that the most sensitive alternative is the one with 
the highest sensitivity coefficient. How to calculate the previous terms is the subject of the next 
section. 

Determining the Threshold Values 'i,j,k 

Case (i): When Using the WSM or the AHP Method 

This theorem provides the main formula used to calculate the threshold values 'i,j,k and it is 
stated next. 

THEOREM 3: When the WSM method is used, the threshold value 'i,j,k (in %) by which the 
performance measure of alternative Ai in terms of criterion Cj, denoted as ai,j, needs to be 
modified so that the ranking of the alternatives Ai and Ak will be reversed, is given as follows: 



'i,j,k < R, when i < k or : 

'i j,k >R when i > k . 

(16a) 



i j,k where R is defined as: 
( Pi-Pk) 100. 

R= x 

Wj aij 

Furthermore, the following condition should also be satisfied for the threshold value to be 
feasible: 

'i,j,k<100. (16b) 

Relation (16b) must hold because from: < aij -'i,j,k the new condition < aij -aijxi,j,k/100 is 
derived which next leads to relation (16b). For the case of the AHP method it can be easily 
shown (see also the Appendix) that the corresponding theorem is as follows: 
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THEOREM 4: When the AHP method is used, the threshold value 'ij,k (in %) by which the 
measure of performance of alternative Ai in terms of criterion Cj needs to be modified so that the 
ranking of alternatives Ai and Ak will change, is given as follows: 

( Pi - Pk ) 100 (17a) 

'i,j,k = x 

[ Pi - Pk + Wj ( akj - aij + 1 aij )] aij 
Furthermore, the following condition should also be satisfied for the threshold value to be 
feasible: 

'i,j,k <100. (17b) 

The sensitivity analysis of the aij values, when the WSM model is used, is next demonstrated in 

terms of a numerical example. 

A Numerical Example for the WSM Case 

Consider the decision matrix depicted in Table 9 (along with the corresponding final preferences 
Pi) of an application of the WSM model (that is, the problem has five alternatives and five 
decision criteria). The AHP case can be developed in an analogous fashion. 
Table 9: Decision matrix and initial preferences for numerical example. 

Criterion 



CI 



C2 



C3 



C4 



C5 



Alt. 0.4146 



0.0129 



0.2958 



0.0604 0.2164 



Pi 



AI 



0.3576 



0.2483 



0.2899 



0.2961 0.3202 



0.3244 



A2 0.3603 



0.2836 



0.0407 



0.0939 0.0172 



0.1745 



A3 0.0255 



0.1745 



0.2895 



0.2212 0.2641 



0.1690 



A4 0.1609 



0.2008 



0.2960 



0.0716 0.0315 



0.1680 



A5 0.0957 



0.0928 



0.0839 



0.3172 0.3670 



0.1643 



Then, when Theorem 4 is used the corresponding J' i,j,k threshold values are as in Table 10. The 
boldfaced entries in Table 10 correspond to the criticality degrees )'ij (i.e., the smallest entry per 
column in each row section, as given in definition 7). The criticality degrees are best 
summarized in Table 1 1 . 

To help interpret the entries in Table 10, consider any one of them, say entry (3,1) (i.e., 
89.3). This entry indicates that '1,1,4 = 89.3%. That is, the measure of performance all must be 
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decreased by 89.3% from its current value (i.e., 0.3576) to (1 - 0.893)x0.3576, in order for 
alternative A4 (which is shown on the left right column in this table) to become more preferred 
than alternative Al (note that currently Al is more preferred than A4). A similar interpretation 
holds for the rest of the entries. Note that some of the entries in the previous table are marked as 
N/F, because they correspond to infeasible values (i.e., condition (17b) in Theorem 4 is violated). 
It can be noticed that in Table 10 entries are greater than 100 only when the sign is negative. 
Recall that negative changes in reality mean increases. If a rating becomes greater than 100, that 
is all right. In the case of criteria weights the numbers will be re-normalized to add up to 1.00. In 
the case of the aij performance measures, the numbers may be re-normalized (for instance, in the 
AHP model) or may become greater than 1.00 (for instance, in the WPM or WSM models). The 
boldfaced numbers indicate minimum values. From Table 11 it follows that the most critical 
alternatives (according to definition 8) are alternatives A3 and A4. This is true because these 
alternatives correspond to the minimum criticality degrees (equal to 1.1) among all values in 
Table 1 1. It can be noticed that the corresponding alternatives from the right column in Table 10 
are now within the parentheses in the entries in Table 11. As before, boldfaced numbers 
represent corresponding minimum values. Finally, Table 12 presents the various sensitivity 
coefficients (as given in definition 9). Note that if in Table 11 was an infeasible entry (denoted 

by the " " symbol), then the corresponding sensitivity coefficient in Table 12 is defined to be 

equal to 0. 



Table 10: Threshold values 'i,j,k (%) in relative terms for Numerical Example. 



Alt. 




Criterion Ci 






Alt. 


(Ai) 


CI 


C2 


C3 


C4 


C5 


Ak 




Al 


74.1 


N/F 


N/F 


N/F 


N/F 


A2 


Al 


N/F 


N/F 


N/F 


N/F 


N/F 


A3 




Al 


89.3 


N/F 


N/F 


N/F 


N/F 


A4 


Al 


96.1 


N/F 


N/F 


N/F 


N/F 


A5 


A2 


-157.9 


N/F 


-1,677.6 


N/F 


N/F 


Al 


A2 


5.4 


N/F 


35.9 


79.4 


N/F 


A3 


A2 


5.3 


N/F 


41.9 


N/F 


N/F 


A4 
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A2 


8.9 


N/F 


78.4 


N/F 


N/F 


A5 


A3 


-1,535.7 


N/F 


N/F 


N/F 


N/F 


Al 


A3 


-39.3 


-355.2 


-8.7 


-52.5 


-13.2 


A2 


A3 


8.0 


38.9 


1.1 


8.3 


2.2 


A4 


A3 


41.1 


N/F 


6.8 


29.9 


7.3 


A5 


A4 


-286.1 


N/F 


N/F 


N/F 


N/F 


Al 


A4 


-8.2 


N/F 


-10.2 


-163.0 


-98.9 


A2 


A4 


-1.7 


-41.4 


-1.1 


-19.7 


-11.5 


A3 


A4 


5.9 


N/F 


5.3 


65.9 


40.4 


A5 


A5 


-460.8 


N/F 


-970.6 


N/F 


N/F 


Al 


A5 


-20.7 


N/F 


-44.4 


-87.4 


-21.2 


A2 


A5 


-12.8 


-544.7 


-15.9 


-29.7 


-6.7 


A3 


A5 


10.0 


N/F 


18.7 


14.9 


3.5 


A4 



Table 11: Criticality degrees) 'ij (%) for each ajj performance measure. 

Alt.(Ai) Criterion Cj 

CI C2 C3 C4 



C5 



Al 



74.1(A2) 



A2 5.3(A4) _ 35.9(A3) 79.4(A3) 

A3 8.0(A4) 38.9(A4) 1.1(A4) 8.3(A4) 2.2(A4) 

A4 1.7(A3) 41.4(A3) -1.1(A3) 19.7(A3) 11.5(A3) 

A5 10.0(A4) 544.8(A3) 15.9(A3) 14.9(A4) 3.5(A4) 

Table 12: Sensitivity coefficients sens(ay) for each aij performance measure in example. 
Alt.(Ai) Criterion Cj 
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CI 



C2 



C3 



C4 



C5 



Al 


0.014(A2) 














A2 


0.189(A4) 





0.028(A3) 


0.013(A3) 





A3 


0.125(A4) 


0.026(A4) 


0.909(A4) 


0.121(A4) 


0.455(A4) 


A4 


0.588(A3) 


0.024(A3) 


0.909(A3) 


0.051 (A3) 


0.087(A3) 


A5 


0.100(A4) 


0.005(A3) 


0.063(A3) 


0.067(A4) 


0.286(A4) 





Case (ii): When Using the WPM Method 

The Appendix also presents the highlights for a proof for Theorem 5. This theorem provides the 
main formula for calculating the threshold values ' i,j,k when the WPM method is used and it is 
stated next. 

THEOREM 5: When the WPM model is used, then the threshold value ' i; j, k (in %) by which the 
performance measure of alternative Ai in terms of criterion Cj, denoted as aij, needs to be 
modified so that the ranking of the alternatives Ai and Ak will be reversed, is given as follows: 



'i,j, k > Q when i > k or : (18a) 
'y,k < Q when i < k. 
and Q is defined as: 
Q = { 1- W N R(a k /aO}xlOO 

Furthermore, the following condition should also be satisfied for the value to be feasible: 
>ij,k<100. (18b) 
The application of the previous theorem is also illustrated in the following numerical example. 
Numerical Example When the WPM Method is Used 

Consider a decision problem which involves the five alternatives Al, A2, A3, A4 and A5 and the 
five decision criteria CI, C2, C3, C4 and C5. Suppose that Table 13 presents its corresponding 
decision matrix and the WPM model is to be used: 
Table 13: Decision matrix for numerical example. 



Criterion 



CI 



C2 



C3 



C4 



C5 



Alt. 



0.2363 0.1998 0.0491 0.2695 0.2453 
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Al 


0.8366 


0.5001 


0.8179 


0.8104 


0.6951 


A2 


0.4307 


0.4782 


0.9407 


0.2062 


0.9259 


A3 


0.7755 


0.5548 


0.6380 


0.3407 


0.0514 


A4 


0.3727 


0.7447 


0.3214 


0.3709 


0.0550 


A5 


0.4259 


0.7126 


0.2195 


0.0470 


0.0014 





Recall that in the WPM method normalization of the aij values is not required. Then, by applying 
formula (4), the current ranking of the alternatives is as shown in Table 14. 

From Table 14 it follows that relation PI >P2 > P3 > P4 > P5 holds and as result the most 
preferred alternative is Al. When Theorem 5 (i.e., formulas (18a) and (18b)) is applied on the 
previous data, then Table 15 with all possible threshold values 'i,j,k is derived. The entries in 
Table 15 have a similar interpretation as the ones in Table 11. 

Table 14: Initial ranking. 



Pair of 






Alternatives 


Ratio 


Ranking 


Ap- 


Aq 


Ap/ Aq 




Al - 


- A2 


1.580 


Al 1 


Al - 


-A3 


2.415 


A2 2 








Al - 


- A4 


2.692 


A3 3 


Al - 


- A5 


6.152 


A4 4 


A2- 


-A3 


1.529 


A5 5 


A2- 


- A4 


1.704 




A2- 


- A5 


3.893 




A3- 


- A4 


1.115 




A3- 


- A5 


2.547 




A4- 


- A5 


2.285 





Table 15: Threshold values 'ij^ (%) in relative terms for WPM example. 
\ 
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Al 


85.6 


89.9 


N/F 


81.7 


84.5 


A2 


Al 


97.6 


98.8 


N/F 


96.2 


97.3 


A3 


Al 


98.5 


99.3 


N/F 


97.5 


98.2 


A4 


Al 


N/F 


N/F 


N/F 


N/F 


N/F 


A5 


A2 


-593 


-887 


-1E+06 


-446 


-546 


Al 


A2 


83.4 


88.0 


N/F 


79.3 


82.3 


A3 


A2 


89.5 


93.0 


N/F 


86.2 


88.6 


A4 


A2 


99.7 


99.9 


N/F 


99.4 


99.6 


A5 



A3 -4,072-8156 -6E+09 -2,538 -3,540 Al 

A3 -502 -736 -6E+05 -383 -464 A2 

A3 36.8 41.9 89.0 33.1 35.7 A4 

A3 98.1 99.1 N/F 96.9 97.8 A5 



A4 -6,501 -14,105 -6E+10 -3,844-5,562 Al 

A4 -853 -1,339 -5E+06 -622 -777 A2 

A4 -58 -72 -811 -50 -56 A3 

A4 97.0 98.4 N/F 95.3 96.6 A5 



A5 -2E+05-9E+05 -1E+18 -8E+04 -2E+05 Al 

A5 -3E+04 -9E+04 -1E+14 -2E+04 -3E+04 A2 

A5 -5,124-10,672 -2E+10 -3,113 -4,420 A3 

A5 -3,202-6,161 -2E+09 -2,049-2,805 A4 



Some entries in Table 15 are represented in standard exponential format. This happens 
because they correspond to very high (negative) values. For instance, the entry (8,1), (i.e., - 
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2E+05) actually represents the value: -2.0x10 = -200,000. It can be argued here that very 
significant changes (such as the ones represented in exponential format or those which measure 
in terms of thousands of % change) are not realistic and practically can also be classified as 
"N/F" (e.g., non-feasible) cases. Finally, observe that the highlighted entries in Table 15 
correspond to the criticality degrees )'ij (as given in definition 7). The criticality degrees are best 
summarized in Table 16. 

Table 16: Criticality degrees ) 'ij (in %) for each aij measure of performance. 
Alt.(Ai) Criterion Cj 



CI 


C2 C3 


C4 


C5 












Al 85.6(A2) 89.9(A2) 




81.7(A2) 


84.5(A2) 


A2 83.4(A3) 88.0(A3) 




79.3(A3) 


82.3(A3) 


A3 36.8(A4) 41.9(A4) 


89.0(A4) 


33.1(A4) 


35.7(A4) 


A4 58.0(A3) 72.0(A3) 




50.0(A3) 


56.0(A3) 


A5 


Table 17: Sensitivity coefficients sens(aij) for each aij measure of performance in example. 


Alt.(Ai) 


Criterion Cj 








CI 


C2 


C3 


C4 


C5 




Al 0.012(A2) 


0.01 1(A2) 





0.012(A2) 


0.012(A2) 


A2 0.012(A3) 


0.01 1(A3) 





0.013(A3) 


0.012(A3) 


A3 0.027(A4) 


0.024(A4) 


0.01 1(A4) 


0.030(A4) 


0.028(A4) 


A4 0.017(A3) 


0.014(A3) 





0.020(A3) 


0.01 8(A3) 


A5 















From Table 16 it follows that the most critical alternative (according to definition 8) is 
alternative A3. This is true because this alternative corresponds to the minimum criticality degree 
(e.g., 33.1) among all values in Table 16. Table 17 presents the various sensitivity coefficients 
(as defined in definition 9). 

Note that if in Table 16 was an infeasible entry (denoted by the " " symbol), then the 

corresponding sensitivity coefficient in Table 17 is defined to be equal to 0. A comparison of 
Tables 11 and 16 (or 12 and 17) indicates that the current WPM example is much more robust 
than the AHP example. This is true because the sensitivity coefficients in the WPM example are 
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much smaller. This is a consequence of the specific data used in these two examples and the 

different nature of the AHP and WPM procedures. 

REMARKS 

The contributions of this paper are both theoretical and empirical. This paper presented a unified 
approach for a sensitivity analysis for three major (and a variant of one) MCDM methods. These 
methods are: the weighted sum model (WSM), the weighted product model (WPM), and the 
analytic hierarchy process (AHP) (both in its original and in ideal mode). The proposed 
sensitivity analysis examines the impact of changes in the weights of importance of the decision 
criteria (i.e., the Wj values) and the measures of performance of the alternatives in terms of a 
single decision criterion at a time (i.e., the ay values) on the final ranking of the alternatives. The 
theoretical contributions of this paper are best summarized in the five theorems presented in the 
previous sections. The empirical contributions are related to the sensitivity analysis of changes in 
the weights of the decision criteria. We did not cover changes on the ay values with an empirical 
study because that would result in too many sensitivity scenarios under consideration for a given 
problem and thus divert the attention from the central ideas. Recall that for a problem with M 
alternatives and N criteria there are MxN different ay values. The two most important empirical 
conclusions of this study are: (i) the choice of the MCDM method or number of alternatives has 
little influence on the sensitivity results; and (ii) the most sensitive decision criterion is the one 
with the highest weight, if weight changes a r measured in relative terms (i.e., as a percentage), 
and it is the one with the lowest weight if changes are measured in absolute terms. The main 
observation of the computational experiments is that the MCDM methods studied here perform 
in similar patterns. These patterns refer to the frequency the criterion with the highest (lowest) 
weight is also the most critical criterion, when changes are measured in percent (absolute) terms. 
Moreover, the same results seem to indicate that the number of decision criteria is more 
important than the number of alternatives in a test problem. 

The proposed methodology can be used to carry out a standard sensitivity analysis when 
one of the previous MCDM methods is used. The benefit of doing a sensitivity analysis is too 
paramount to be ignored in applications of MCDM techniques to real-life problems. As Dantzig 
(1963, p. 32) stated it: "Sensitivity analysis is a fundamental concept in the effective use and 
implementation of quantitative decision models, whose purpose is to assess the stability of an 
optimal solution under changes in the parameters." By knowing which data are more critical, the 
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decision maker can more effectively focus his/her attention to the most critical parts of a given 
MCDM problem. Another area of application is during the phase of gathering the data for a 
MCDM problem, given a limited budget. Often, in real-life applications of MCDM, data are 
changeable and cannot be precisely determined. In such cases it makes more sense to determine 
with higher accuracy the weights of the criteria (as well as the aij measures of performance) 
which are more critical and with less accuracy the less critical weights. A sensitivity analysis, 
contacted at an early stage, may reveal which Wj and ay values have a tendency to be more 
critical to the final decisions. Therefore, these data can be determined with higher accuracy at a 
second stage. Next, a new sensitivity analysis cycle can be initiated again. This process can be 
repeated, in this stepwise manner, for a number of times until the entire budget is used or the 
decision maker is satisfied with the robustness of the results. The three (and a variant of one) 
MCDM methods examined in this paper have been fuzzified by Triantaphyllou and Lin (1996). 
Thus, a natural extension of this research is to develop a sensitivity analysis approach for cases 
in which the data are fuzzy numbers. An additional area of possible extension is to extend these 
results to AHP problems with multiple hierarchies. 
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